Automatically Annotating the ODP Web Taxonomy

In this paper we present the ideas and algorithms developed around our KeyGen Web Taxonomy Annotation engine. KeyGen annotates the Open Directory Project, also known as Dmoz, with meaningful and previously unknown keywords by utilizing domain knowledge extracted from the WWW. We present two algorithms: i) The PageParse Algorithm, which efficiently extracts keywords from Web Taxonomies using a combination of local and global scores, and ii) the Support Algorithm, an I/O optimized algorithm for coalescing hierarchies of keywords. We then present the results: i) from constructing a richly annotated ODP Web taxonomy and ii) from evaluating the correctness of this structure by performing an automated classification of Web-pages.